Search AI Products and News

Recently, the DeepSeek team released a technical paper about their latest model, DeepSeek-V3, focusing on the challenges of scaling in large AI model training and related considerations of hardware architecture. This 14-page paper not only summarizes the experiences and lessons learned during the development of V3 by DeepSeek but also provides deep insights for future hardware design. Notably, CEO of DeepSeek, Liang Wenfeng, also participated in writing the paper.

Wikipedia recently announced the release of a dataset optimized for AI model training, in collaboration with Kaggle, Google's data science community platform. This initiative aims to reduce the scraping of Wikipedia data by AI developers, conserving the platform's bandwidth and server resources. The dataset includes structured Wikipedia information in English and French, offering high machine readability for AI developers to facilitate modeling, fine-tuning, and data analysis. The Wikimedia Foundation stated that this data...

Meta Platforms, Inc. has announced plans to use user data from its European Union applications, including Facebook and Instagram, to train its artificial intelligence models. The company clarified that the training data will include users' public posts, comments, and interactions with Meta AI, but will exclude private messages with friends and family. Training will be limited to users aged 18 and over. Meta stated it will inform its EU users of this plan this week via in-app notifications and emails.

Amidst the growing prevalence of artificial intelligence, the collaboration between DeepSeek and Tsinghua University has garnered significant industry attention. DeepSeek, a Chinese startup, is renowned for its breakthroughs in low-cost inference models. This collaboration aims to further reduce the training costs of AI models, thereby enhancing operational efficiency. DeepSeek recently launched a new low-cost inference model that has generated considerable market excitement. To further optimize this model, DeepSeek's research team...

Bloomberg, citing sources, reports that Ant Group has made a significant breakthrough in AI, reducing AI model training costs by 20% using chips from Alibaba and Huawei. This injects new momentum into China's drive for technological self-reliance. The report further indicates that Ant Group's internal tests show these Chinese-made AI chips rival Nvidia's offerings in performance. Widespread validation and adoption of this could significantly alter the global AI landscape.

GPU cloud services company CoreWeave recently announced a five-year, $119 billion strategic partnership agreement with OpenAI. Under the agreement, CoreWeave will provide OpenAI with the necessary computing power to support the training and delivery of its AI models. This significant deal marks a deepening collaboration between the two companies in the field of artificial intelligence. As part of the collaboration, CoreWeave will also issue $3...

ByteDance's Doubao large model team recently announced a breakthrough in addressing key bottlenecks in Mixture-of-Experts (MoE) architecture, open-sourcing a significant optimization technology called COMET. This technology dramatically improves large model training efficiency, achieving a remarkable 1.7x speedup and a 40% reduction in training costs. Image Note: Image generated by AI, image licensing provider Midjourney. COMET has been deployed in ByteDance's multi-thousand-GPU cluster training, resulting in millions of GPU hours saved.

In today's rapid development of Large Language Models (LLM), the training and inference costs of models have increasingly become a focus of research and application. Recently, the Tencent Hunyuan team released an important study that delves into the 'Scaling Laws' of low-bit floating point quantization training, which refers to the principles governing the scale of floating point quantization training. The core of this research lies in exploring how to significantly reduce computational and storage costs without sacrificing performance by lowering the precision of the model.

Earlier this year, Google released the sixth generation of TPU, the Trillium, which is the most powerful TPU to date. Today, Trillium is officially available for Google Cloud customers. Google has trained its latest AI model, Gemini 2.0, using the Trillium TPU, making it the strongest AI model created by Google so far. Now, enterprises and startups can leverage the same robust, efficient, and sustainable infrastructure. The core of the AI supercomputer: Tri

Large Language Models (LLMs) like GPT and Llama have sparked a revolution in the field of artificial intelligence, but efficiently training these massive models while aligning them with human values remains a challenge. Reinforcement Learning with Human Feedback (RLHF) has become an important training method for LLMs in recent years, but traditional RLHF frameworks have limitations in flexibility, efficiency, and scalability. To address these issues, the ByteDance Doubao Large Model Team has open-sourced an RLHF framework called HybridFlow.

Recently, ByteDance officially addressed rumors regarding an 'intern disrupting large model training'. The company confirmed that an intern did maliciously interfere with the model training tasks of the commercialization technology team's research project. However, this action did not affect the company's official commercial projects and online business, nor did it involve other large model businesses of ByteDance. Moreover, ByteDance pointed out that claims circulating online about 'involving over 8,000 nodes and causing losses of over ten million dollars' are seriously exaggerated. According to public reports, the intern was part of ByteDance's commercialization technology team.